A computing device is often carried along by a user throughout the day. The computing device can be used by the user to provide mapping information, navigational directions, and to search for establishments such as stores and restaurants. Furthermore, the computing device can utilize speech recognition to perform one or more tasks requested by the user's spoken commands. Often times, there can be sounds, such as background noise, associated with the current environment in which the user and the computing device are situated. The sounds can vary depending on the current environment of the user and computing device. For example, the sounds of a restaurant can be quite different from the sounds of an airport. In another example, a particular auditorium can sound differently when different events are occurring at the auditorium. Because the types of sounds vary widely, it is difficult to accurately filter out background noise without risk of inadvertently filtering some of the intended audio input.